RESNA Annual Conference - 2010

BACKGROUND

Powered Wheelchair (PW) users suffer from impairments that may affect their ability to navigate safely through an environment. A study shows that more than 10% of the PW users may benefit from some degree of technological assistance to facilitate wheelchair independence (1). Various prototypes of autonomous wheelchairs have been developed. However, design and experimentation have often focus on healthy subjects, usually the engineers or students involved in the development (2,3,4,5). Often, the design process did not take into account the perspective of users. Moreover, there is a lack of consistent experimental procedure between different projects to test PW skills, limiting the ability to compare prototypes (6).

This project drew together researchers in engineering and rehabilitation, clinicians, technicians and PW users to help develop a prototype with multiple interface devices. This prototype takes into consideration the needs expressed by potential users in terms of their independence in mobility and activities of daily living (ADL). The Wheelchair Skills Test (7) provides a test-bench for evaluating the functionality of the prototype with both healthy subjects and PW users. The participation of researchers in rehabilitation and clinicians provided guidance in defining the requirements for the various autonomous functions, the structure and content of experiments, and the supervision of the validation process.

The goal of this paper is to present the overall design of the platform, the initial validation process with members of the interdisciplinary team and the on-going experimental validation procedure with rehabilitation professionals, technicians and PW users.

INTELLIGENT POWERED WHEELCHAIR DESIGN

Design Goal Overview

The intelligent wheelchair (see Figure 1) was designed with the following goals in mind:

1. Formal assessment of the needs and potential benefits of autonomous navigation for PW users who present various physical and neurological disabilities;
2. Portable autonomous navigation system that can be installed on standard PWs with minimum modification to the manufactured wheelchair;
3. Robust autonomous system that guarantees user’s safety and comfort;
4. Robust and natural interface for controlling the intelligent wheelchair;
5. Formal assessment of the performance of the intelligent system in a representative set of wheelchair skills.

Autonomous Navigation Modules

The intelligent control system of the autonomous module is shown in Figure 2. Based upon sensing information about the environment provided by laser and sonar range-finders, odometric data provided by wheel rotation sensors, and the navigation goal as specified through the multimodal interface, the control system directs execution of the corresponding maneuver modules, using a distributed and layered architecture (8).  

The maneuver modules provide autonomous execution of a set of basic tasks included in (or adapted from) the Wheelchair Skills Test (7). These include moving along a straight line for a specified distance, moving along an ascending or a descending slope, following a specific structure (such as a wall) in the environment, maneuvering laterally so as to move away from a structure (a wall for example), to near another one, following a person or another wheelchair. The module in charge of each maneuver may involve multiples behaviours varying according to the task requirements, and consisting of specific wheelchair kinematic (speed, acceleration) profiles selected to achieve smooth and jerk-free motion. Behaviours are adjusted in order to account for local environment structure as well as user’s overall condition and capabilities. All above maneuvers are executed under the supervision of a collision avoidance module that overrides their normal execution through deceleration, avoidance or halt until the obstacle disappears from the motion path. Reference speeds are sent to the Dynamic Controller which produces the required motor commands. This controller executes a closed loop control on linear and angular speed separately by using two proportional-integral-derivative compensations on the actual wheelchair speeds computed from odometry measurements. This type of control is essential to allow crossing of difficult terrain sections such as gravel and slopes and to prevent deviations from a set course.  Extensive testing of the autonomous navigation modules shows that they are robust in dynamic contexts and achieve safe and efficient displacements in constrained environments.

User Interface

The interface architecture provides the mechanism for interaction between the user and the wheelchair. Voice commands provide a convenient mode of interaction. However speech controlled systems traditionally monitor the speech recognizer for a list of known keywords to activate specific functionalities of the intelligent systems. This tends to be very brittle and sensitive to the speech recognition performance, as well as requiring the user to memorize a set of commands. The goal of our system is to provide robustness to noise in the recognition system, as well as allow the user to speak naturally to the wheelchair, without strong vocabulary restrictions. To handle this, an architecture (9) which takes advantage of modern intelligent system techniques is used.

The Speech Recognition component is handled by Dragon Naturally Speaking, a commercially available software package used for speech to text transcription. Users perform a 10 minute training session which allows the system to adapt to their speech patterns. After training, the user can speak to the system which then attempts to transcribe the audio signal into text. 

The output of the speech recognizer is processed by a natural language parser, which extracts semantic information from the transcribed text and removes lexical and syntactic details. For natural language parsing we use a combinatory categorical grammar (10), for which an open-source implementation is freely available (11).

A key component of our interface is the Interaction Manager. The purpose of the Interaction Manager is to infer the user’s true intention (despite errors in speech recognition) and select appropriate commands for the autonomous navigation system. To achieve this, the Interaction Manager uses probabilistic models and a decision-theoretic algorithm (9). These models are able to incorporate the noise and ambiguity from the speech recognition, as well as context from previous actions.  As information is received from the natural language parser, the interaction manager adjusts its belief in the user’s intention, and selects an appropriate response accordingly. The system can choose to respond by requesting that the user provide more information or repeat the previous command. Alternately, the system can choose to issue a command to the wheelchair’s autonomous navigation system.

Helpful feedback is provided to the user through a 7” LCD display mounted on the wheelchair. The system displays either the command being currently executed by the wheelchair, or a screen requesting additional information from the user.

VALIDATION PROCESS

The validation process consists of two parts: 1) an initial validation with team members and 2) the experimental validation with rehabilitation professionals, technicians as well as PW users. The initial validation is completed and preliminary results are discussed below. The experimental validation procedure is also presented and testing with PW users and rehabilitation professionals is on-going.

Initial validation

During the preliminary testing, nine members of the team (students, engineers, clinicians, technicians and one PW user) evaluated the IPW. The objectives were: 1) to identify standardized mobility tasks and skills required to perform ADL 2) to adapt the design of the IPW and the interfaces to perform these tasks safely 3) to test the reliability and performance of the navigation system and the multimodal interfaces, and 4) to identify the benefits and limits of the IPW and the interfaces.

While the long-term goal of the project is to develop and validate an IPW platform for a variety of living environments, initial evaluations are focusing on a more constrained task domain using the standardized environment of the Wheelchair Skills Test (WST). Version 4.1 for PW users (see www.wheelchairskillsprogram.ca ) includes 32 representative wheelchair skills (e.g. transfers, moving forward, maneuvering sideways, rolling on side slope, rolling on 5° incline, transfers into and out of wheelchair) performed in their ADL, indoor and in the community (7).

Experimental validation

The objectives of the experimentations are: 1) to demonstrate the ability to move around independently using the wheelchair’s intelligent functions (avoid obstacles, follow a wall, park independently, etc); 2) to establish that the IPW can be reliably and safely controlled using the multimodal interfaces (vocal interface, PC joystick or custom keyboard); 3) to collect comments and suggestions in order to improve the prototype; and 4) to identify the target clientele and functionalities that are essential for potential users. The experimental aspect of the validation process will be performed in two parts. Part One involves 12 assistive technology providers (occupational therapists, educators, mechanics and orthotics and prosthetics technicians) working with PW users. For each subject, the testing procedure involves three sessions: two sessions will be used to administer the WST with the multimodal interface (objectives 1 & 2) and the last session will consist of a participation in focus groups (objective 3 & 4). Part Two of the validation process involves 12 adults PW users, 4 in each group of disabilities involving a stable neurological condition (e.g. stroke, spinal cord injury), a slow degenerative disorder (e.g. multiple sclerosis, ataxia) and a musculoskeletal impairment (e.g. rheumatoid arthritis), receiving services of the Assistive Devices Department of the 2 participating rehabilitation centers in Montreal. The inclusion criteria are the abilities to: speak both French and English, use a vocal command and participate in a half-day evaluation session. Exclusion criteria’s are: the presence of dysarthria, rapid degenerative conditions, emotional and psychiatric impairments, cognitive deficits and the use of a complex positioning system. WST testing will be performed first in conventional mode (with a standard joystick), than in intelligent mode with the vocal interface (objectives 1 & 2), followed by an interview (objectives 3 & 4). Two scales are used in the WST to assess each skill on a pass or fail score: one scale evaluates the performance and the second the ability to perform the skill safely.

PRELIMINARY RESULTS AND DISCUSSION

During the preliminary testing, the members of the team identified 20 tasks of the WST that test the desired navigation functions of the IPW. The remaining 12 consist mainly of skills related to the user only (i.e. selecting driving modes, transfers, pressure relief). Two specific tasks, the ascent and descent of the 5 cm threshold and crossing the 15 cm pothole, were removed from the experimentation due to potential hazards resulting from the intense vibration to the computer hardware and the inability, for now, to detect negative obstacles in the latter task. The standardized environment of the WST allowed the team to evaluate ongoing progress of the navigation system and the multimodal interface.

The IPW, controlled at first through a joystick and keyboard connected to the on-board computer, was able to perform most of the tasks related to the WST while using the different autonomous modules developed for specific tasks. Users felt secure and thought that the autonomous navigation system was robust. The main complaint related to the delayed response time caused by the use of a computer joystick rather than the conventional PW joystick, but users were able to quickly adapt to this delayed response. Overall, few collisions or potentially unsafe situations, caused by sensor misalignments or blind spots for example, were observed.

Preliminary trials of the vocal system show adequate recognition of most speech commands and PW response to the specified navigation tasks. The use of such an interface required a longer training period in order to adapt to the PW behaviors. Team members were able to use vocal commands to navigate and perform WST tasks using the autonomous navigation strategies. Factors contributing to reduce the recognition rate were: difficulties in the training with non native English speaking users, increased ambient noise, emotional states (i.e. impatience or frustration). Furthermore, natural speaking was encouraged with the use of Dragon recognition system in order to improve performance.

CONCLUSION

An interdisciplinary approach implemented early in the development of the IPW has been key to ensure that the design is oriented towards targeted autonomous navigation skills and that the interfaces are user friendly.  Preliminary feedback from team members has allowed further improvement of the overall, reliability and safety in order to begin experimentation with IPW targeted users in the environment representative of their ADL mobility tasks.